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Continuous Improvement In California Education: 



Data Systems and Policy Learning 



The “Getting Down to Facts” (GDTF) studies released in March 2007 offered a clear diagnosis of 
the issues facing California’s education system. Now, as California moves beyond the facts and 
begins the search for ways to improve the performance of California schools and students, the 
state faces a critical policy dilemma. On the one hand, the evidence presented in GDTF made it 
clear that simply putting more resources into California’s present education system is unlikely to 
produce the large gains in performance that Californians expect from their schools. On the other 
hand, the GDTF studies made it equally clear that bringing about significant improvements in 
educational performance may require a substantial increase in the resources that the state spends 
on education, along with increased autonomy and flexibility for local educators to decide how 
these resources should be used. The policy dilemma that the state faces is how to ensure that local 
actors use new resources in the best possible ways, without increasing the regulatory burden on 
schools and school districts or adding to the profusion of categorical funding streams. 

In PACE's view, the solution to this dilemma has two key elements. First, the state needs to 
focus its reform efforts on creating a system that fosters innovation and learns from experience to 
support continuous improvement toward the goal of academic success for all students. The 
critical first step toward this goal is to accelerate current efforts to build a strong and 
comprehensive data system based on the collection and analysis of longitudinal data on individual 
students and teachers. Second, the state needs to make significant investments in human capital 
and capacity building at all levels of the education system. Personnel policies must ensure that 
California educators have the time, knowledge, and skill they need to improve the performance of 
their schools and students, and incentives within the education system should be aligned to 
encourage the development and adoption of new and more effective practices. 

We address the first of these two elements in this policy brief. 1 We argue that a robust data 
system is the essential foundation for an education system that is capable of continuous 
improvement in school and student performance. Great data by themselves will not lead to 
continuous improvement in the educational performance, however. Policies must also be 
designed and implemented in ways that support careful evaluation and the production of new 
knowledge about effective programs and practices. In addition, the state must support the 
creation of institutions to disseminate new knowledge and encourage the adoption of best 
practices at all levels of the education system, from the classroom to the California Department of 
Education (CDE). 



1 We address the second element in another PACE policy brief, “Reshaping Personnel Policies to Improve 
Student Achievement,” by Julia E. Koppich and Amy Gerstein. 




I. 



Data Collection 



A strong data system that produces rich and timely data on students, classrooms, and schools is a 
necessary condition for the creation of a continuously improving education system. The 
availability of reliable data is essential to the accurate measurement of performance at all levels 
of the system, and to the identification of practices, programs, and policies that are effective in 
improving the performance of schools and students. Timely and reliable data can support 
teachers as they make decisions about the students in their classrooms, and principals as they 
make decisions about the allocation of resources in their schools. District and state officials need 
comprehensive data to inform their decisions about policy and funding. Taxpayers and voters 
need far more information than they currently have on what’s working and what’s not in 
California’s education system so that they can make informed decisions about how best to 
support California’s students and secure the state's economic future. In the absence of timely and 
reliable data, California’s education system is literally flying blind. 

California’s education data system lags far behind data systems in other states and other 
countries, generally failing to provide the kinds of information that teachers, principals, parents, 
and others need to support continuous improvement in the performance of schools and students. 

The Legislature should: 

• Move immediately to accelerate the implementation of C ALP ADS and CALTIDES, 
including sufficient funds for districts to support the training and infrastructure necessary 
to ensure that data collection is both timely and reliable. 

• Expand the scope of data collection under CALPADS and CALTIDES to focus on 
identifying policies and programs that enhance learning outcomes, and not simply on 
compliance with federal mandates. The marginal cost of collecting additional 
information while the system is under development is low; expanding the scope of data 
collection later could prove costly. 

• Ensure that data on individual teachers and students can be linked, to identify what 
teaching practices and strategies are working and to target support to students and 
teachers who need it. 

• Support the development of a data system in which education data can be linked to data 
from other sectors, including but not limited to higher education, pre-K education, social 
services, health care, corrections, and employment, in order to understand how factors 
outside of schools affect student performance and also to track the impact of educational 
policies and programs into students' adult lives. Student learning is affected by multiple 
factors inside and outside the school, and it is important for California’s education data 
system to be able to account for as many of these factors as possible. 

• Consider basing unique student and teacher identifiers on social security numbers, w/ 
appropriate safeguards, as is already done in other states, in order to make linkages across 
data systems possible. 

II. Data Use 

Funding alone may not be enough to gain the buy-in at the local level needed to ensure the quality 
of data collection. School and district personnel must also see how their investments in the data 
system support their own efforts to improve the performance of local schools and students. Too 
often, educators perceive data as a threat, widely used to stigmatize and punish them and rarely 
used to support them or expand their opportunities for success. The perception that data are 




dangerous is a powerful disincentive to the creation of a strong and effective data system, because 
those responsible for collecting and using the data may have an interest in seeing the system fail. 
Changing this perception will require the state to return the data that it collects to teachers, 
parents, schools, and school districts on timelines and in formats that support their efforts to 
improve learning outcomes for the students under their care. In addition to quality and 
timeliness, the state must also provide infrastructure and training to support data analysis and use 
at all levels of the education system. 

The Legislature should: 

• Support efforts by CDE and school districts to develop timely and informative “report 
cards” on the performance of individual schools. Parents need information on the schools 
where their children are enrolled not in the middle of the year, when their children are 
already deeply enmeshed in relationships and activities, but before the school year begins 
when they are still in a position to make decisions about the schools they would like their 
children to attend. 

• Support efforts by CDE and school districts to make data on student performance 
available to teachers at times when they can use it. Teachers need data on the 
performance of the students in their classes not at the end of the year — when these 
students are about to become someone else’s responsibility — but at the beginning of the 
year when information about students’ strengths and weaknesses might enable teachers to 
adapt their instruction to students’ needs. 

• Invest in capacity-building to support data collection, data analysis, and informed 
decision-making at all levels of the system. Investments in capacity building will have to 
include the employment of specialized personnel, training and professional development 
for educators, and increased time for educators to make use of the data available to them. 

• Support the creation of state-level institutions for the accumulation and validation of 
information on “best practices,” including the development of effective mechanisms for 
making knowledge about effective programs and practices available to teachers, school 
leaders, and district officials. At present lessons learned in one school or district typically 
remain the exclusive property of those directly involved, because there are few channels 
through which new knowledge can be validated or diffused. The Legislature should seek 
to ensure that learning that occurs in one part of the system (e.g., one school district, or 
one teacher’s classroom) is made available to others within the system. 

TIT. Continuous Improvement 

Policy implementation concerns are not separate from the data system. Even with great data and 
access, it will be difficult for Californians to learn which policies and programs are working 
unless those policies and programs are implemented in ways that facilitate evaluation. The data 
system is needed in order to provide the information necessary to evaluate programs, but 
deliberate implementation is also needed in order to ensure the availability of informative data. 
The state must also invest resources in evaluation to learn which policies and practices are 
effective in improving performance and which are not. 

The Legislature should: 

• Design and implement policies in ways that support organizational learning. This might 
often involve the design and implementation of quasi-experiments, in which new policies 
and practices are adopted in a carefully selected sample of schools and classrooms, in 




order to identify whether and under what circumstances new approaches result in better 
outcomes for students, before they are put into practice in all schools. 

• At the state level, commit resources to rigorous, independent policy evaluation, to 
identify programs and practices that improve the performance of California schools and 
students. It is not sufficient to encourage innovation and experimentation in the 
education system. It is also necessary to evaluate carefully and systematically how new 
policies and practices affect (or not) academic performance. Without careful evaluation, 
the opportunities for learning afforded by new flexibility are likely to be squandered. 

• At the district and school level, provide resources (time, training, and specialized 
personnel) to increase capacity to support data analysis, data use, and organizational 
learning. Educators at all levels of the system need time and support to reflect on their 
work, to consider different ways of organizing and carrying out their responsibilities, and 
to provide support and guidance for others. For example, teachers not only need timely 
data on the past and current performance of their students; they also need training to be 
able to interpret the data and determine what the data have to tell them about which 
practices will be most effective with which students. 

IV. Measuring Improvement 

Judging the success or failure of schools solely on whether their students meet the California 
grade-level standards now ensures that most schools that enroll large numbers of poor students 
will be labeled as failures. As longitudinal data on students and teachers become available 
California will therefore need to develop additional “value-added” measures that can fairly and 
accurately assess the progress that schools are making toward the goal of proficiency for all of 
their students, in order to take fair account of the different challenges faced by different schools. 

• Valid measures of student progress are not supported by California’s current data system, 
but they will become increasingly feasible with the full implementation of CALPADS 
and CALTIDES. Agreement on the nature of these measures and how they should be 
incorporated into California’s accountability system should be a priority for policy- 
makers. 

• Psychometricians in California and across the country are currently at work on the 
development of valid indicators of student progress, which will allow policy-makers and 
officials to measure the “value added” by different instructional programs and strategies. 
The Legislature can support this work by providing incentives to school districts and 
union locals that are willing to experiment with these new technologies by putting them 
to work to support continuous improvement in the performance of schools, teachers, and 
students. 

V. Conclusion 

California has set very high goals for the performance of our state’s education system, and the 
current performance of the system is a long way from meeting those goals. As we challenge our 
state’s educators and students to do better, though, it is essential to recognize that the goal we 
seek — a system that educates all students, including the most disadvantaged, to very high 
standards — far exceeds what any education system in the U.S. currently achieves. Learning how 
to accomplish this goal is not simply a matter of learning what other states are doing and 
imitating high performers; it requires learning how to educate students in new and better ways. 

To accomplish California’s ambitious educational goals, the state’s education system will have to 
be reorganized to support innovation and to learn from experience. 




California does not currently have the capacity to use information well. In order to do so, the 
state must systematically and consistently collect data on students, teachers, classrooms, schools 
and districts. The data that the state collects must be synthesized and distributed in ways that 
make them useful to stakeholders, and made available for independent evaluations. In addition, 
the state should begin to implement policies and programs in ways that support evaluation and 
organizational learning. Without strong support for data collection, data use, and policy learning 
in the education system the goals that we have set for our state’s schools and students will remain 
out of reach. 

VI. Research and Additional Resources 

Policy Analysis for California Education (PACE) is a non-partisan policy research center based at 
the University of California - Berkeley and Stanford University. PACE seeks to increase the 
impact of academic research in educational policy debates in California. The policy 
recommendations included in this brief are based on the research reported in “Getting Down to 
Facts,” and on continuing research at PACE and elsewhere on data systems and the conditions 
required for continuous improvement in educational systems. PACE will publish two additional 
policy briefs on data systems and continuous improvement in California’s education system in 
Fall 2007. 




